The devices, systems, and methods disclosed herein relate generally to marker-based tool tracking, and more particularly, to devices, systems, and methods that are configured to perform marker-based tool tracking in ophthalmic surgeries.
Surgical tools, such as surgical imaging probes, surgical forceps, surgical scissors, surgical vitrectomy probes, and the like, may be inserted into an eye during an ophthalmic surgery to perform various surgeries in the eye. Typically, a distal portion of a surgical tool is inserted into the eye during the ophthalmic surgery. Thus, the area of the eye surrounding the distal tip of the surgical tool is a region of interest to a surgeon. To achieve guided surgical interventions, such as intra-operative Optical Coherence Tomography (OCT) for Internal Limited Membrane (ILM) peeling, automatic tool tip tracking is used to efficiently close a feedback loop to allow the OCT engine to locate the scanning target area. Further, to provide real time feedback during surgery, useful surgical data may be overlaid to the surgeon's current area of interest. When a surgeon moves the distal portion of the surgical tool inserted into the eye, the area of interest may shift accordingly. Thus, automatic tool tracking may be used to locate the area of interest to adjust the surgical data overlay so the surgeon may visualize it without looking away from the current area of interest.
There are three conventional techniques for general object tracking. The first technique is motion-based object tracking. Motion-based object tracking may be used for automated surveillance. Motion-based object tracking may use image processing algorithms, such as background subtraction, frame difference, and optical flow, to track an object. Nevertheless, motion-based object tracking algorithm requires a quasi-stationary background and may not be suitable for tool tracking in an ophthalmic surgery in which background may vary constantly.
The second technique for general object tracking is region-based object tracking. Region-based object tracking may be used for tracking simple objects. In region-based object tracking, an object template is preselected offline or during a first frame. For the subsequent frames, the template is searched across the whole field of view and the location with the greatest similarity to the template is identified as the object. Nevertheless, region-based object tracking is sensitive to object pose variations and local illumination changes and may not be suitable for tool tracking in an ophthalmic surgery, in which illumination and orientation of the tool vary greatly.
The third technique for general object tracking is feature-based object tracking. Feature-based object tracking may extract and search for unique features of an object, such as contour, edge, shape, color, corner/interest point and the like, across the entire field of view for object detection. Nevertheless, in the feature-based tracking algorithm, a high contrast feature, which is not sensitive to environmental and object pose changes and is unique to the object, is required. Since most of the surgical tools do not possess high contrast intrinsic features, feature based object tracking may not provide suitable results.
In a vitreo-retinal surgery, illumination conditions may be challenging for tool tracking. An endo illuminator may be inserted into the eye for illumination. Because the endo illuminator may move during a surgery, the illumination condition may vary greatly from image frame to image frame and the images of the fundus area being illuminated may change greatly over time. Motion-based and region-based object tracking techniques may be difficult to implement under inconsistent illumination conditions. Further, with a single illuminator illuminating from one side, shadow artifacts and specular reflection from the surgical tool may increase complexity for tool tracking. Moreover, in order to capture a fundus image through a video camera, a beam path of an imaging light from the eye may pass through multiple optical elements and media, such as eye vitreous body, an aged crystalline lens, eye cornea, and Binocular Indirect Ophthalmomicroscope (BIOM) lenses. These optical elements in the beam path of imaging light may further degrade the image quality and reduce contrast. Thus, it may be difficult to extract an intrinsic feature of various surgical tools to achieve real time tool tracking.
The present disclosure is directed to devices, systems, and methods that address one or more of the disadvantages of the prior art.